2016-09-30

Be more intelligent with your `sleep`--or-- how to overengineer your scripts

The universal solution to waiting for something to be ready in shell scripting is the sleep command.

Here, we’re waiting for a dir to be created (say from a yum install httpd going on in another terminal), so we can ls the contents, perhaps as part of a script that configures httpd.

sleep 6
ls /etc/init.d/httpd

But is there a better way? What if the directory exists almost immediately? You’ve wasted nearly 6 seconds unnecessarily, which if you do this a lot in your script, adds a bunch of time.

We need a way to continuously query if the resource exists. And it needs to be with a command that sets an error exit code if the resource is not found. Let’s use stat to do this.

Leveraging the fact that stat will set a non-zero exit code on failure

while true; do
stat /etc/init.d/httpd
if [ $? -eq 0 ]; then # check if the stat was successful
break
fi
done
ls /etc/init.d/httpd

‘true‘ is a command, whose output is nothing and whose exit status is 0. In terms of performance we’d probably like not to call a binary, but true is a shell built-in. The no-op operator : accomplishes the same thing, ex: while :; do.

We can simplify the if statement by using the && operator which executes the following command break, if stat exits without error (sets a 0 status)

while true; do
stat /etc/init.d/httpd && break
done
ls /etc/init.d/httpd

Instead of stat, we can use the test command (aliased as [). Here we check for a file using -f.

while true; do
if [ -f /etc/init.d/httpd ]; then break; fi
done
ls /etc/init.d/httpd

However, in these examples, if the file never exists, the loop will never exit.

So, instead we can define a timeout and a shorter sleep interval, and a counter (i) to track the iterations:

i=0
while [ “$i” -lt 6 ]; do
if [ -f /etc/init.d/httpd ]; then break; fi
sleep 1
(( i++ )) # built-in arithmetic
done
ls /etc/init.d/httpd

Quote i for protection. Use the break keyword to escape the while loop.

Alternatively, use Bash’s built-in arithmetic:
i=0
while (( i < 6 )); do
if [ -f /etc/init.d/httpd ]; then break; fi
sleep 1
(( i++ ))
done
ls /etc/init.d/httpd

This is a good start. Calling test (even as a builtin) in an infinite loop is also wasteful. If you are in bash, you can use the [[ keyword, which has the added benefit of protecting you against unquoted variables in a comparison.

i=0
while (( i < 6 )); do
if [[ -f /etc/init.d/httpd ]]; then break; fi

sleep 1
(( i++ ))
done
ls /etc/init.d/httpd

There is a bug here. ls will run no matter whether the file was found or not.

So now we exit the loop, but how do we notify the caller that it failed? The break statement does not return a non-zero. As far as the shell is concerned, the loop completed. We can use a RETVAL variable that we set explicitly to 0 when it succeeds, and 143 to mean “file not found”.

i=0
while (( i < 6 )); do
if [[ -f “$path” ]];then
RETVAL=0
else
RETVAL=143
fi

[[ “$RETVAL” -eq 0 ]] && break

sleep 1
(( i++ ))
done
[[ “$RETVAL” -eq 0 ]] && ls /etc/init.d/httpd || echo “ERR: file never created”

I quote variables in the [[ ]] here even though it’s not actually required.

Which works fine, but we can simplify by simply unsetting RETVAL if there’s a success. The test is expressed with [[ -z $xxx ]]. Also it is best practice to send error messages to STDERR (file descriptor –or fd– #2), using >&2.

i=0
while (( i < 6 )); do
if [[ -f “$path” ]];then
unset RETVAL
else
RETVAL=143
fi

[[ -z “$RETVAL” ]] && break

sleep 1
(( i++ ))
done
[[ -z “$RETVAL” ]] && ls /etc/init.d/httpd || echo “ERR: file never created” >&2

Now, let’s abstract the variables, and set an exit status.

i=0; path=”/etc/init.d/httpd”; timeout=6
while (( i < $timeout )); do
if [[ -f “$path” ]]; then
unset RETVAL
else
RETVAL=143
fi

[[ -z “$RETVAL” ]] && break

sleep 1
(( i++ ))
done
[[ -z “$RETVAL” ]] && ls “$path” || (echo “ERR: file never created” >&2; exit $RETVAL )

Notice that the exit is called from a subshell, as if it is called in the current context, it will exit your interactive shell which is annoying and undesirable. There is no way to use return in this context since this is not yet in a function.

Testing with a bogus file.

$ path=/etc/bogusfile

$ while (( i < $timeout )); do

> if [[ -f “$path” ]]; then
> unset RETVAL
> else
> RETVAL=143
> fi

> [[ -z “$RETVAL” ]] && break

> sleep 1

> (( i++ ))

> done

$ [[ -z “$RETVAL” ]] && ls “$path” || (echo “ERR: file never created”; exit $RETVAL)

ERR: file never created

$ echo $?

143

Now put it in a reusable function!

_testforfile () {
local i=0
local timeout=”$1”
local path=”$2”

while (( i < $timeout )); do
if [[ -f “$path” ]];then
unset RETVAL
else
local RETVAL=143
fi

[[ -z “$RETVAL” ]] && break

sleep 1
(( i++ ))
done
[[ -z “$RETVAL” ]] && ls “$path” || (echo “ERR: file never created” >&2; return “$RETVAL” )
}

Note the use of the

</span><span style="font-family: &quot;courier new&quot; , &quot;courier&quot; , monospace;">local</span><span style="font-family: inherit;">

keyword so we don’t have our custom variables pollute the invoking environment, and the change of exit to return. This function accepts two parameters as input. Call it like so:

_testforfile 6 /etc/init.d/httpd

Now, we can enhance this by making sure that at least the first argument is numeric, and even set that to a default value of 5 if it was not provided.

_testforfile () {
local i=0
local timeout=“${2:-5}”
local path=”$1”

local re=’^[0-9]+/span>
if ! [[ $timeout =~ $re ]]; then
echo “ERR: Timeout was not a number” >&2
return 1
fi

while (( i < $timeout )); do
if [[ -f “$path” ]];then
unset RETVAL
else
local RETVAL=143
fi

[[ -z “$RETVAL” ]] && break

sleep 1
(( i++ ))
done
[[ -z “$RETVAL” ]] && ls “$path” || (echo “ERR: file never created” >&2; return “$RETVAL” )
}

re is a regular expression used in conjunction with the =~ operator.

In action:
$ _testforfile 47d /etc/hosts
ERR: Timeout was not a number
And maybe some tests in a future post.

Why aren't things better?

modern-day user interactions at their worst

Be more intelligent with your `sleep`--or-- how to overengineer your scripts